Digital imaging systems have created a revolution in photography and cameras. A digital camera is similar to a film camera except that the film is replaced with an electronic sensor. The sensor is comprised of an array of photo detectors that change the photons that strike them into electrons providing a signal at each pixel proportional to the number of photons, or the amount of light at each pixel. Presently, most consumer digital cameras employ Charge Coupled Device (CCD) or Complementary Metal Oxide Semiconductor (CMOS) sensors. To facilitate the collection of light, many of the sensors employ a small lens-like structure covering each pixel, which is called a microlens. These microlenses are typically made by forming a layer of photoresist that is placed upon the pixel plane.
The image sensors used in digital imaging are inherently monochrome devices, having no color discriminating ability associated with each detector. For this reason, the sensors typically employ a Color Filter Array (CFA) inserted between the microlens and each active portion of the pixel structure, the photodiode. Typically, the CFA is constructed to assign a single color to each pixel. Digital camera manufacturers often choose among a variety of CFA architectures, usually based on different combinations of primary colors (red, green, blue) or complementary colors (cyan, magenta, yellow). Regardless of the particular CFA used, the overall aim is to transfer only a single color of interest, so that each pixel sees only one color wavelength band.
One of the most popular and ubiquitous CFA patterns is called the Bayer Pattern, which places red, green and blue filters over the pixels, in a checkerboard pattern that has twice the number of green squares as red or blue. The theory behind the Bayer Pattern is that the human eye is more sensitive to wavelengths of light in the green region than wavelengths representing red and blue. Therefore, doubling the number of green pixels provides greater perceived luminance information and detail, while natural color representation for the human eye.
When subjected to light, the image sensor photo diode converts incident photons to electrons. This conversion enables analog electronic circuitry to process the image “seen” by the sensor array. The electrons gathered by these sensors are stored in small capacitors that are read out as a series of varying voltages, which are proportional to the image brightness. An Analog to Digital Converter (ADC) conditions these voltages for processing by a computer within the camera. The data is processed to form a picture of the image “seen” by the sensor array. The sensor is only one part of the digital camera. An associated processor controls the sensor and processes the data from the sensor to achieve high quality output images. The tasks performed by the processor include Automatic Exposure Control (AEC), Automatic White Balance (AWB), color interpolation or “demosaicing”, color correction, contrast enhancement or gamma control, and noise reduction. A properly tuned sensor/processor combination results in superior digital camera images.
In many designs, a mechanical shutter is used in the same way that it is used in a film camera—to gate the light that is allowed to reach the sensor or film. Many digital cameras use what is called an electronic shutter, which allows the control of when and for how long the sensor gathers light through electronic control signals to the sensor. Proper design of the automatic exposure control algorithm ensures high quality images.
In a single sensor camera using a color filter array, one type of algorithm implemented by the processor is called a color interpolation or demosaicing algorithm, which is used to derive complete color data for each pixel. Demosaicing algorithms analyze neighboring pixel color values to determine the resulting color value for a particular pixel, thus delivering a full resolution image that appears as if each pixel's color value was derived from a combination of the red, blue, and green primary colors (if RGB colors are used). Thus, the assembled image can exhibit natural gradations and realistic color relationships.
Automatic white balance (AWB) algorithms are used to set the overall color ‘tone’ of a digital image, correcting for different colors of illumination and for color biases inherent in the sensor. Other types of algorithms specifically for color correction allow the digital data to be further processed to achieve a particular color and intensity (or shade) associated with a specific pixel. Some set a default saturation level high, producing extremely bright, sometimes unnatural colors. Others choose a neutral, more realistic saturation, for greater subtlety and color accuracy. A gamma correction algorithm is used in many digital cameras to tune the image histogram, attempting to make details in both light and dark portions of the image visible.
As digital imaging becomes more prevalent, industry is striving to develop images and video with better resolution and color accuracy. The quality of an image produced by a digital camera can be measured in several ways such as resolution and dynamic range. In the most basic sense, the resolution of a digital camera can be stated to be the total number of pixels used. For example, a camera with a sensor that has 800 by 1000 pixels can be said to have 800,000 pixels of total resolution. As stated, the resolution is a quality of the camera mechanism. In an image, the resolution is the limit of the ability to depict fine lines, sharp edges, and other details in the original scene. Resolution depends on the optical system (lens, filters, etc), the number of pixels, and the manner of processing the digital signal.
The dynamic range relates to both the accuracy of the A/D converter that changes the analog voltage representing a pixel brightness to a digital number. Typical cameras have A/D converters of 10 to 12 bits or 1024 to 4096 different dissemble levels. Better cameras may have higher A/D resolution. The dynamic range is the largest possible signal containing information, divided by the smallest possible signal that is distinguishable from noise, usually expressed in decibels.
The broader the dynamic range of the camera, the easier it is to differentiate small gradations in brightness or color. Noise can come from several different places including optical noise from lens flare, thermal and electronic noise from the power supply, electronic circuitry and the sensor itself. Unique to digital cameras, a large component of the noise is thermally generated within the sensor. Cooling the sensor can reduce this noise but electronically cooling the sensor with a Thermo-Electric (TE) cooler is both expensive and consumes quite a bit of power. In total, the dynamic range will interpret into a digital camera's ability to observe fine gradations in the darkest shadow areas of an image or equivalently in the brightest highlights.
In taking a digital image, one aim is to achieve images in which objects are exposed properly and are not too bright or too dark. Commonly, this is achieved through moving the digital information histogram to an optimal point within the bounds of the maximum and minimum output signal levels of the system. Another related aim is to achieve images that are properly exposed and which exhibit a high signal-to-noise ratio. Stated another way, the aim is to acquire properly-exposed images that do not show many noise artifacts.
There is a continuing need to develop improved digital imaging systems and processes to capture higher quality images and video. It is possible to capture higher quality images and video through improving the digital processes that manipulate the raw data acquired from the photo sensor array. In addition, it is possible to improve the operation of digital image and video systems through streamlining the overall process architecture. CMOS sensors, having many cost, power, and circuit advantages over CCD sensors, tend to have poorer signal-to-noise and dynamic range than CCDs. Improved digital processes are of great value in avoiding unnecessary noise in the final image from a CMOS sensor, and making best use of its available dynamic range.